Yioop v10 Mail Refactor — Plan

Arc Post-Mortem Summary. This arc moved the mail logic out of the oversized SocialComponent into MailModel and the mail library and backend classes. listMessages was unified onto the backends, per-user session state (sort, sequence cache, sort-support) moved to MailModel, default-folder resolution and the unread-count walk moved onto MailBackend so SocialComponent issues no raw IMAP, and address and header parsing and validation became library and model helpers. The result is a much smaller SocialComponent with mail concerns separated out.

Two parts, in order. Fixed list; each step only gains a ✓ when done. ▶ marks the step in progress. Ongoing in every patch (not a step): inline single-use methods under 20 lines and leave no stubs in SocialComponent.

1. Part 1 — move mail logic out of SocialComponent into MailModel and the mail library classes
   1. ✓ listMessages unified onto the backends
   2. ✓ MailModel — per-user session state (sort, sequence cache, sort-support)
   3. ✓ default-folder resolution moved onto MailBackend
   4. ✓ unread-count walk moved into the backend (SocialComponent now issues no raw IMAP)
   5. ✓ address/header parsing and validation into a library/model: userMailParseAddressList, userMailExtractBareAddress, userMailIsValidAddress, userMailCleanHeader, userMailFilterAddressList, userMailJoinAddressLists, userMailCleanEnvelopes, userMailEnvelopeSender
   6. ✓ account-form parse/validate into MailAccountModel: userMailParseAccountFields, userMailValidateAccountFields, userMailCleanAccountFormValues
      Reviewed → kept in the controller. These are controller concerns with no model/library home: parse reads the request, validate produces tl() messages, and the clean is display-prep. The model stores raw values (the IMAP/SMTP client needs raw HOST/USERNAME to connect) and the controller escapes them for the view at render time. A first attempt that moved these onto the model was reverted — the same disposition as userMailCleanEnvelopes / userMailFromOptions.
   7. ✓ compose / reply / forward / MIME building into a library builder: userMailBuildReplyPrefill, userMailReplySubject, userMailFromOptions, userMailCleanComposeValues, userMailSendIdentities
      Done: new MailComposeBuilder (replySubject + replyPrefill); userMailBuildReplyPrefill kept only the backend fetch and delegates the building. userMailSendIdentities → MailAliasModel::identitiesFor. userMailFromOptions and userMailCleanComposeValues stay in the controller — they call $parent->clean, which a library class must not.
   8. ✓ folder-list session cache into MailModel: userMailReadCachedFolders, userMailRememberFolders, and the MAIL_FOLDERS_CACHE session state with its invalidation points
      Done: MailModel gained cachedFolders (fresh-or-null), cacheFolders (write + freshness stamp), invalidateFolders, foldersFor (raw read for the cross-account move and folder-protected check), and allCachedFolders (the FOLDERS_BY_ACCOUNT map); the MAIL_FOLDERS_CACHE_TTL const moved too. SocialComponent now issues no direct $_SESSION["MAIL_FOLDERS"] access. Part 1 complete.
2. Part 2 — consolidate the mail library classes into src/library/mail/ (namespace seekquarry\yioop\library\mail; MailSite.php stays in atto; BulkEmailJob / MailCloneJob stay in media_jobs)
   1. ✓ relocate and re-namespace, updating all references: ImapClient, ImapMailBackend, MailBackend, MailSiteMailBackend, MailSiteFactory, SmtpClient, MimeMessage, ImapEnvelopeParser, ImapFolderListParser, ImapResponseParser, ImapListing, DkimKey, MailHeaderParser, MailRecordCache, MailScheduledDispatcher, MailUnreadProbe, MailBackendException (and MailComposeBuilder, added in 1.7)
      In progress. MailHeaderParser relocated first as the template (a leaf, no outbound mail-class deps). Established procedure per class: create src/library/mail/, git mv the file, change its namespace to seekquarry\yioop\library\mail, then update every reference — change the use import in importing files, and add a use in former same-namespace library classes that had relied on same-namespace resolution. The autoloader maps the namespace straight to the path, so no loader change is needed. Done so far: MailHeaderParser (the template), then leaves MailComposeBuilder and MailRecordCache (the latter adding a use to its former same-namespace users DkimKey, SpfCheck, DmarcCheck). The final 15 (ImapClient, the Imap parsers, ImapListing, ImapMailBackend, MailBackend, MailSiteMailBackend, MailSiteFactory, SmtpClient, MimeMessage, DkimKey, MailScheduledDispatcher, MailUnreadProbe, MailBackendException) then moved in a single pass: moving them together kept every mail-to-mail reference same-namespace, so only external referencers changed — use imports repointed, the L\ library-alias mail uses rewritten to a new ML (= library\mail) alias, one fully-qualified reference fixed, and the now-redundant same-namespace imports dropped. All 18 mail classes now live under src/library/mail/; MailSite.php stays in atto and BulkEmailJob / MailCloneJob in media_jobs. The mail email-auth checks DmarcCheck and SpfCheck (both DB-clean) were relocated too, so the whole src/library/mail/ set is in place. Part 2 complete.
3. Part 3 — library layering cleanup: nothing in src/library may touch a model or the database. Models are the only layer that touches the database; controllers and components are the only layers that touch models. Library classes must take their data as arguments. Mail-related violations to fix (move the DB/model access into a model, have the library class receive values as parameters):
   1. MailSiteFactory instantiates MailAliasModel, UserModel, MailSenderAllowModel directly. Coupled with the authenticator item below: both are set up in MailSiteFactory::build(), which is called from the MailServer executable, SocialComponent, and two library classes (MailSiteMailBackend, MailCloneJob). The library callers cannot create models to inject, so this needs the daemon-layer decision before it can be fixed.
   2. ✓ MailBackend and MailUnreadProbe no longer reach a model. Done: the component loads the account row (new SocialComponent::userMailBackend helper) and passes it to MailBackend::forAccountId; the unread probe takes an accounts-provider closure so the lookup happens only on a cache miss; new model method MailAccountModel::getAccountsWithPassword.
   3. ✓ CredentialCipher no longer touches the database. Done: the key store moved into MailAccountModel::masterKey() (loads or mints the MAIL_SECRET key, caches it on the model); the cipher is now pure crypto whose encrypt/decrypt take the raw key as a parameter. The key-persistence test moved from CredentialCipherTest into MailAccountModelTest.
   4. ✓ YioopUserAuthenticator relocated into src/library/mail/. Done: folder move plus namespace change. This also repaired a latent break — MailSiteFactory (already in library\mail) referenced new YioopUserAuthenticator() unqualified, which had been resolving to a non-existent library\mail\YioopUserAuthenticator since the factory itself moved; the class now lives there, so the reference resolves. Its model-instantiation layering fix is still deferred with the factory below.
   5. MailSiteFactory — left as-is for now (per Chris), apart from the authenticator now resolving from mail/. Still instantiates MailAliasModel, UserModel, MailSenderAllowModel; resolving that needs the daemon-layer decision (build() is called from the MailServer executable, a component, and two library classes that cannot inject models).
   6. Note: non-mail library classes violate this too (FeedDocumentBundle, VersionFunctions, several media jobs); out of scope for the mail arc but flagged here